1. Field of the Invention
The present invention relates to a signal processing device, an imaging device, and a signal processing method. In particular, the present invention relates to a technique of interpolating color information to each pixel signal obtained through a color filter array, for example.
2. Description of the Related Art
In a single-plate type imaging device, a color filter array is used so as to separate object light obtained through a lens into three primary colors of R (red), G (green), and B (blue). A Bayer array is commonly used as the color filter array. An example of the Bayer array is illustrated in FIG. 13. In the Bayer array, G pixels to which a luminance signal contributes at a high rate are arranged checkerwise and R pixels and B pixels are respectively arranged checkerwise in the rest part of the array.
Since each pixel can obtain data of only one color among R, G, and B, data of other colors, which are not obtained, are obtained by performing an interpolation calculation using outputs of surrounding pixels. In a case of a position of R33 in FIG. 13, missing G component and B component are interpolated by the calculation.
As a method of the interpolation, such method is disclosed that, among pixels positioned in proximity to a pixel of interpolation object (also referred to below as a pixel of interest), interpolation is performed by using only a pixel in a direction in which the pixel is estimated to have strong correlation with the pixel of interest (for example, refer to J. F. Hamilton and J. E. Adams: “Adaptive color plane interpolation in single sensor color electronic camera,” U.S. Pat. No. 5,629,734). In this method, an amount of change in a pixel value of each of pixels, which are positioned in proximity to the pixel of interest, as compared to a pixel value of the pixel of interest is often used as a barometer representing strength of correlation. Accordingly, the interpolation is performed in such a manner that a direction in which an amount of change in a pixel value is small is considered as a direction of high correlation.
The amount of change in a pixel value is often estimated based on an amount of change in a pixel value of G. This is because G pixels have more amount of information than R pixels and B pixels due to the Bayer array in which G pixels are arranged in the largest number. That is, accuracy in determination of the strong-correlation direction can be improved by determining a strong-correlation direction based on the amount of change in a pixel value of G.
However, in a case where the pixel of interest is positioned on a corner or a texture part of an image, a direction in which pixels having strong correlation with the pixel of interest are positioned (referred to below as a correlation direction) may not be correctly estimated by determination based on the amount of change in a pixel value of G. This is because there are a plurality of directions in which the amount of change in a pixel value of G is high, at a corner or a texture part.
D. Cok: “Signal Processing Method and Apparatus for Producing Interpolated Chrominance Values In a Sampled Color Image Signal,” U.S. Pat. No. 4,642,678, 1987, for example, discloses a method which uses not only a spatial correlation of a specific primary color (G, for example) but also a correlation between primary colors. This method is based on a hypothesis: “a color component does not suddenly change in a local region” (constant hue hypothesis). Namely, it is assumed that an amount of change in R (or B) (referred to below as R/B) and an amount of change in G (a ratio of R/B and a ratio of G (referred to below as a color ratio)) are nearly equal to each other due to a correlation between different primary colors. Based on this assumption, a color ratio of each of pixels surrounding around the pixel of interest is produced and a color ratio of the pixel of interest is estimated from the color ratio of the surrounding pixels so as to estimate an interpolation value of the pixel of interest.
Further, such a method is disclosed that a color component of surrounding pixels which is estimated to have high correlation is largely weighted depending on an amount of change in a pixel value of G and an amount of change in a pixel value of the pixel of interest (R or B) so as to estimate a color component of the pixel of interest (For example, refer to H A. Chang and H. Chen: “Directionally Weighted Color Interpolation for Digital Cameras,” Circuits and Systems, 2005. ISCAS 2005. IEEE International Symposium on 23-26 May 2005, Page(s): 6284-6287 Vol. 6, and Japanese Unexamined Patent Application Publication No. 2006-174485). According to this method, color interpolation can be performed while suppressing an unnatural artifact (false color) even in an edge in which a color suddenly changes.
However, in the above-described method, detection may be less-accurately performed by using amounts of change in pixel values of pixels of a single color and estimation is performed by using less-accurate color components of surrounding pixels. Accordingly, in a texture part having a high frequency and the like, accuracy in estimating a color component of a pixel of interest is disadvantageously degraded when surrounding pixel values are not produced (interpolated) based on a correct estimation.
Therefore, such a method is disclosed that correlation-direction detection is repeated after such color interpolation processing, thus improving accuracy in estimating a color component of surrounding pixels of the pixel of interest, and detecting a correlation direction with higher accuracy so as to re-constitute a color interpolation image (for example, refer to R. Kimmel, “Demosaicing: Image reconstruction from CCD samples, “IEEE Trans. Image Processing, vol. 8, pp. 1221-1228, 1999).